Autonomous vehicle, control system for remotely controlling the same, and method thereof

ABSTRACT

An autonomous vehicle may include a display device configured to display a driving path of the autonomous vehicle; and an autonomous driving control apparatus including a processor that displays a situation in which driving of the driving path is impossible on the display device in augmented reality when the situation in which the driving of the driving path is impossible occurs due to an external environment during autonomous driving of the autonomous vehicle, transmits information related to a misrecognized obstacle to a control system when receiving a request for deleting the misrecognized obstacle, and receives a driving path in which the misrecognized obstacle is deleted from the control system and controls and follows the received driving path.

CROSS-REFERENCE TO RELATED APPLICATION

The present application claims priority to Korean Patent Application No.10-2021-0086050, filed on Jun. 30, 2021, the entire contents of which isincorporated herein for all purposes by this reference.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to an autonomous vehicle, a control systemfor remotely controlling the same, and a method thereof, and moreparticularly, to a technique for correcting an error occurring in aremote control situation of an autonomous vehicle.

Description of Related Art

As an electronic technique of a vehicle develops, an interest in anautonomous vehicle that drives to a destination by recognizing a drivingenvironment of the vehicle itself without manipulation of a driver isgrowing more and more.

An autonomous vehicle refers to a vehicle capable of operating by itselfwithout manipulation of a driver or a passenger.

While driving in an autonomous driving mode, there may be a situation inwhich it is impossible to follow a driving path to the destinationnormally although there is no abnormality in a function of the vehicle.Accordingly, when a situation where it is impossible to follow a pathoccurs during autonomous driving of the autonomous vehicle, it is oftendifficult to follow the driving path, such as when the driver directlyintervenes in control of the vehicle or when the driver's interventionis difficult, the vehicle stops.

The information disclosed in this Background of the Invention section isonly for enhancement of understanding of the general background of theinvention and may not be taken as an acknowledgement or any form ofsuggestion that this information forms the prior art already known to aperson skilled in the art.

BRIEF SUMMARY

Various aspects of the present invention are directed to providing anautonomous vehicle, a control system for remotely controlling the same,and a method thereof, configured for securing reliability for remotecontrol by correcting a surrounding situation recognition error throughremote control during remote control of the autonomous vehicle,improving commercialization of autonomous driving.

Furthermore, various aspects of the present invention are directed toproviding an autonomous vehicle, a control system for remotelycontrolling the same, and a method thereof, configured for increasinguser convenience by displaying a change in a remote control path suchthat a driver can intuitively check it when the remote control path ischanged.

The technical objects of the present invention are not limited to theobjects mentioned above, and other technical objects not mentioned maybe clearly understood by those skilled in the art from the descriptionof the claims.

Various aspects of the present invention are directed to providing anautonomous vehicle, including a display device configured to display adriving path of the autonomous vehicle; and an autonomous drivingcontrol apparatus including a processor that displays a situation inwhich driving of the driving path is impossible on the display device inaugmented reality when the situation in which the driving of the drivingpath is impossible occurs due to an external environment duringautonomous driving of the autonomous vehicle, transmits informationrelated to a misrecognized obstacle to a control system when receiving arequest for deleting the misrecognized obstacle, and receives a drivingpath from which the misrecognized obstacle is deleted from the controlsystem and controls and follows the received driving path.

In various exemplary embodiments of the present invention, theautonomous vehicle may further include a sensing device configured todetect the situation in which the driving of the driving path isimpossible.

In various exemplary embodiments of the present invention, the processormay display the driving path in which the misrecognized obstacle isdeleted received from the control system on the display device in theaugmented reality.

In various exemplary embodiments of the present invention, the processormay display an area in which the misrecognized obstacle is deleted inthe driving path and controls the area to blink for a predetermined timeperiod.

In various exemplary embodiments of the present invention, the processormay mark or block an area in which a vehicle is unable to be driven inthe driving path.

In various exemplary embodiments of the present invention, theprocessor, when the driving path is displayed on the display device inthe augmented reality, may display an area of a front target formaintaining an in-vehicle distance with a vehicle in front of theautonomous vehicle, and may distinguish and displays a line thickness ora line color indicating the area of the front target during a normaldriving mode and a stop control mode of the autonomous vehicle.

In various exemplary embodiments of the present invention, the processormay display a front signal condition during the stop control mode.

In various exemplary embodiments of the present invention, the processormay display an area of the misrecognized obstacle in the driving path,and may display a line color or line thickness indicating the area ofthe misrecognized obstacle separately from that of the normal drivingmode.

In various exemplary embodiments of the present invention, the processormay transmit information related to the misrecognized obstacle to thecontrol system when receiving a request for deleting the misrecognizedobstacle from the driver.

In various exemplary embodiments of the present invention, theprocessor, when receiving no request for deleting the misrecognizedobstacle from the driver, may request remote control of the autonomousvehicle to the control system.

In various exemplary embodiments of the present invention, the processorwhen receiving a remote control path for the remote control request fromthe control system, may display the remote control path in the augmentedreality.

In various exemplary embodiments of the present invention, the processormay display a screen for obtaining approval for the remote control pathby a driver or an occupant on the display device.

In various exemplary embodiments of the present invention, the processormay follow and control the remote control path when approval for theremote control path from the driver or the occupant is completed.

In various exemplary embodiments of the present invention, the processormay transmit unapproved information and an unapproved remote controlpath to the control system when the approval for the remote control pathis not completed.

In various exemplary embodiments of the present invention, theautonomous driving control apparatus may further include: acommunication device configured to communicate with the control system;and a storage configured to store the remote control path received fromthe control system.

Various aspects of the present invention are directed to providing acontrol system including a processor configured to display amisrecognized obstacle on a driving path of an autonomous vehicle whenreceiving information related to the misrecognized obstacle from theautonomous vehicle, deletes the misrecognized obstacle on the drivingpath of the autonomous vehicle by receiving approval from an operator,and transmits a driving path of the autonomous vehicle in which themisrecognized obstacle is deleted to the autonomous vehicle.

In various exemplary embodiments of the present invention, the processormay generate a remote control path to provide the remote control path tothe autonomous vehicle when receiving a request for remote control ofthe autonomous vehicle for avoiding the misrecognized obstacle from theautonomous vehicle.

In various exemplary embodiments of the present invention, the processormay re-generate a remote control path for avoiding the misrecognizedobstacle when approval for the remote control path is not received fromthe autonomous vehicle.

Various aspects of the present invention are directed to providing aremote control method for an autonomous vehicle, including: displaying asituation in which driving of a driving path is impossible on a displaydevice in augmented reality when the situation in which the driving ofthe driving path is impossible occurs due to an external environmentduring autonomous driving; receiving a request for deleting amisrecognized obstacle from a driver; transmitting information relatedto the misrecognized obstacle to a control system; and following andcontrolling a driving path in which the misrecognized obstacle isdeleted by receiving the driving path from the control system.

In various exemplary embodiments of the present invention, displayingthe driving path in which the misrecognized obstacle is deleted, whereinthe displaying includes displaying an area in which the misrecognizedobstacle is deleted, and controlling the area to blink for apredetermined time period.

According to the present technique, it is possible to secure reliabilityfor remote control by correcting a surrounding situation recognitionerror through remote control during remote control of the autonomousvehicle, improving commercialization of autonomous driving.

Furthermore, according to the present technique, it is possible toincrease user convenience by displaying a change in a remote controlpath such that a driver can intuitively check it when the remote controlpath is changed.

Furthermore, various effects which may be directly or indirectlyidentified through the present specification may be provided.

The methods and apparatuses of the present invention have other featuresand advantages which will be apparent from or are set forth in moredetail in the accompanying drawings, which are incorporated herein, andthe following Detailed Description, which together serve to explaincertain principles of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a block diagram showing a configuration of a remotecontrol system for an autonomous apparatus according to variousexemplary embodiments of the present invention.

FIG. 2A illustrates a view for describing a sensing device of anautonomous vehicle according to various exemplary embodiments of thepresent invention.

FIG. 2B illustrates a sensing range of a sensing device of an autonomousvehicle according to various exemplary embodiments of the presentinvention.

FIG. 3 illustrates an example of a screen displaying an obstacle inaugmented reality in a normal driving mode in an autonomous vehicleaccording to various exemplary embodiments of the present invention.

FIG. 4 illustrates an example of a screen displaying an obstacle inaugmented reality in a stop control mode in an autonomous vehicleaccording to various exemplary embodiments of the present invention.

FIG. 5 illustrates an example of a screen displaying an obstacleincluding traffic light information in augmented reality in a stopcontrol mode in an autonomous vehicle according to various exemplaryembodiments of the present invention.

FIG. 6 illustrates an example of a screen displaying an obstacleincluding traffic light information which displays a misrecognizedtarget in augmented reality in a misrecognition mode in an autonomousvehicle according to various exemplary embodiments of the presentinvention.

FIG. 7 and FIG. 8 illustrate examples of a screen for deleting amisrecognized obstacle depending on a driver request according tovarious exemplary embodiments of the present invention.

FIG. 9 illustrates an example of a screen on which a misrecognizedobstacle is displayed depending on a driver request in a control systemaccording to various exemplary embodiments of the present invention.

FIG. 10 illustrates an example of a screen displaying a change in aremote control path in augmented reality in an autonomous vehicleaccording to various exemplary embodiments of the present invention.

FIG. 11 illustrates a flowchart showing a remote control method for anautonomous vehicle according to various exemplary embodiments of thepresent invention.

FIG. 12 illustrates a computing system according to various exemplaryembodiments of the present invention.

It may be understood that the appended drawings are not necessarily toscale, presenting a somewhat simplified representation of variousfeatures illustrative of the basic principles of the present invention.The specific design features of the present invention as includedherein, including, for example, specific dimensions, orientations,locations, and shapes will be determined in part by the particularlyintended application and use environment.

In the figures, reference numbers refer to the same or equivalent partsof the present invention throughout the several figures of the drawing.

DETAILED DESCRIPTION

Reference will now be made in detail to various embodiments of thepresent invention(s), examples of which are illustrated in theaccompanying drawings and described below. While the presentinvention(s) will be described in conjunction with exemplary embodimentsof the present invention, it will be understood that the presentdescription is not intended to limit the present invention(s) to thoseexemplary embodiments. On the other hand, the present invention(s)is/are intended to cover not only the exemplary embodiments of thepresent invention, but also various alternatives, modifications,equivalents and other embodiments, which may be included within thespirit and scope of the present invention as defined by the appendedclaims.

Hereinafter, some exemplary embodiments of the present invention will bedescribed in detail with reference to exemplary drawings. It should benoted that in adding reference numerals to constituent elements of eachdrawing, the same constituent elements have the same reference numeralsas possible even though they are indicated on different drawings.Furthermore, in describing exemplary embodiments of the presentinvention, when it is determined that detailed descriptions of relatedwell-known configurations or functions interfere with understanding ofthe exemplary embodiments of the present invention, the detaileddescriptions thereof will be omitted.

In describing constituent elements according to various exemplaryembodiments of the present invention, terms such as first, second, A, B,(a), and (b) may be used. These terms are only for distinguishing theconstituent elements from other constituent elements, and the nature,sequences, or orders of the constituent elements are not limited by theterms. Furthermore, all terms used herein including technical scientificterms have the same meanings as those which are generally understood bythose skilled in the technical field to which various exemplaryembodiments of the present invention pertains (those skilled in the art)unless they are differently defined. Terms defined in a generally useddictionary shall be construed to have meanings matching those in thecontext of a related art, and shall not be construed to have idealizedor excessively formal meanings unless they are clearly defined in thepresent specification.

Hereinafter, various exemplary embodiments of the present invention willbe described in detail with reference to FIG. 1 to FIG. 12 .

FIG. 1 illustrates a block diagram showing a configuration of a remotecontrol system for an autonomous apparatus according to variousexemplary embodiments of the present invention.

Referring to FIG. 1 , the remote control system for an autonomousvehicle according to various exemplary embodiments of the presentinvention includes a vehicle 100 and a control system 200, and remotecontrol may be performed through communication between the vehicle 100and the control system 200. In the instant case, the vehicle 100 mayinclude an autonomous vehicle.

The vehicle 100 may include an autonomous driving control apparatus 120,a sensing device 120, a steering control apparatus 130, a brakingcontrol apparatus 140, and an engine control apparatus 150.

The autonomous driving control apparatus 110 according to the exemplaryembodiment of the present invention may be implemented inside thevehicle. In the instant case, the autonomous driving control apparatus110 may be integrally formed with internal control units of the vehicle,or may be implemented as a separate device to be connected to controlunits of the vehicle by a separate connection means.

The autonomous driving control device 110 may request remote control tothe control system 200 when a situation occurs in which driving of adriving path is impossible due to an external environment duringautonomous driving.

In the instant case, when the situation in which the driving of thedriving path is impossible occurs due to misrecognition of an obstacledetected by the sensing device 120, the autonomous driving controlapparatus 110 may delete the misrecognized obstacle.

Furthermore, the autonomous driving control apparatus 110 may displaythe situation in which the driving of the driving path is impossible andthe misrecognized obstacle based on augmented reality such that a drivercan check it at a glance.

When receiving a request for deleting the misrecognized obstacle fromthe driver, the autonomous driving control apparatus 110 may transmitinformation related to the misrecognized obstacle to the control system200, and may continue to perform autonomous driving by receiving theremote control path in which the misrecognized obstacle is deleted fromthe control system 200.

In the instant case, the autonomous driving control apparatus 110 maydetermine that the driver has requested the deletion of themisrecognized obstacle when the driver double-touches the misrecognizedobstacle or presses and holds the misrecognized obstacle for apredetermined time period.

Furthermore, when the driver does not request the deletion of themisrecognized obstacle, the autonomous driving control apparatus 110 mayrequest the remote control for avoidance control for the misrecognizedobstacle to the control system. Accordingly, the autonomous drivingcontrol apparatus 110 may receive a remote control path for avoidancecontrol for the misrecognized obstacle from the control system 200 tocontinue the autonomous driving.

Referring to FIG. 1 , the autonomous driving control apparatus 110 mayinclude a communication device 111, a storage 112, an interface device113, and a processor 114.

The communication device 111 is a hardware device implemented withvarious electronic circuits to transmit and receive signals through awireless or wired connection, and may transmit and receive informationbased on in-vehicle devices and in-vehicle network communicationtechniques. As an example, the in-vehicle network communicationtechniques may include controller area network (CAN) communication,Local Interconnect Network (LIN) communication, flex-ray communication,Ethernet communication, and the like.

Furthermore, the communication device 111 may perform communication byuse of a server, infrastructure, or third vehicles outside the vehicle,and the like through a wireless Internet technique or short rangecommunication technique. Herein, the wireless Internet technique mayinclude wireless LAN (WLAN), wireless broadband (Wibro), Wi-Fi,Worldwide Interoperability for Microwave Access (WiMAX), Ethernetcommunication, etc. Furthermore, short-range communication technique mayinclude Bluetooth, ZigBee, ultra wideband (UWB), radio frequencyidentification (RFID), infrared data association (IrDA), and the like.For example, the communication device 111 may perform wirelesscommunication with the control system 200, may transmit vehicle positioninformation (e.g., vehicle coordinates), surrounding information (e.g.,obstacle information), vehicle information (e.g., overall length andwidth of a host vehicle), a remote control request, etc. to the controlsystem 200, and may receive a remote control path, an approval requestfor the remote control path, a remote control command, and the like fromthe control system 200.

The storage 112 may store sensing results of the sensing device 120,information received from the control system 200, data and/or algorithmsrequired for the processor 114 to operate, and the like.

As an example, the storage 112 may store vehicle information, a vehicledriving path, front image data captured by a camera, and a remotecontrol path received from the control system 200.

The storage 112 may include a storage medium of at least one type amongmemories of types such as a flash memory, a hard disk, a micro, a card(e.g., a secure digital (SD) card or an extreme digital (XD) card), arandom access memory (RAM), a static RAM (SRAM), a read-only memory(ROM), a programmable ROM (PROM), an electrically erasable PROM(EEPROM), a magnetic memory (MRAM), a magnetic disk, and an opticaldisk.

The interface device 113 may include an input means for receiving acontrol command from a user and an output means for outputting anoperation state of the autonomous driving control apparatus 110 andresults thereof. Herein, the input means may include a key button, andmay further include a mouse, a keyboard, a touch screen, a microphone, ajoystick, a jog shuttle, a stylus pen, and the like. Furthermore, theinput means may further include a soft key implemented on the display.

The output means may include a display, and may further include a voiceoutput means such as a speaker. In the instant case, when a touch sensorformed of a touch film, a touch sheet, or a touch pad is provided on thedisplay, the display may operate as a touch screen, and may beimplemented in a form in which an input device and an output device areintegrated.

In the instant case, the display may include at least one of a liquidcrystal display (LCD), a thin film transistor liquid crystal display(TFT LCD), an organic light emitting diode display (OLED display), aflexible display, a field emission display (FED), or a 3D display.

As an example, the interface device 113 may be implemented as a head-updisplay (HUD), a cluster, an audio video navigation (AVN), a humanmachine interface (HM), a user setting menu (USM), or the like.

For example, the interface device 113 may display the remote controlpath received from the control system 200, the approval request for theremote control path, and the remote control command.

Furthermore, the interface device 113 may receive an approval input froma driver or an occupant on an approval request screen for a remotecontrol path received from the control system 200. To the present end,the interface device 113 may receive the input from the driver through amouse, a keyboard, a touch screen, a microphone, or the like.

The processor 114 may be electrically connected to the communicationdevice 111, the storage 112, the interface device 113, and the like, mayelectrically control each component, and may be an electrical circuitthat executes software commands, performing various data processing andcalculations described below.

The processor 114 may process a signal transferred between components ofthe autonomous driving control apparatus 110, and may perform overallcontrol such that each of the components can perform its functionnormally.

The processor 114 may be implemented in a form of hardware, software, ora combination of hardware and software, or may be implemented asmicroprocessor, and may be, e.g., an electronic control unit (ECU), amicro controller unit (MCU), or other sub controllers mounted in thevehicle.

When a situation in which driving of a driving path is impossible due toan external environment occurs during autonomous driving, the processor114 may display the situation in which the driving of the driving pathis impossible on the interface device 113 in the augmented reality.Furthermore, when receiving a request for deleting a misrecognizedobstacle that causes the situation in which the driving of the drivingpath is impossible from the driver, the processor 114 transmit theinformation related to the misrecognized obstacle to the control system,and may receive the driving path in which the misrecognized obstacle isdeleted from the control system 200 and follow and control the receiveddriving path.

The processor 114 may perform a remote control request for avoidancecontrol of the misrecognized obstacle to the control system 200 whenreceiving no request for deleting the misrecognized obstacle from thedriver.

The processor 114 may transmit information for remote control when theremote control is requested to the control system 200. In the instantcase, the information may include vehicle position information (e.g.,vehicle coordinates), image information around the vehicle, informationaround the vehicle (e.g., obstacles, moving vehicle information,stationary vehicle information (fixed objects), map information, and thelike.

The processor 114 may display the driving path in which themisrecognized obstacle received from the control system 200 is deletedon the interface device 113 in the augmented reality. That is, theprocessor 114 may display an area in which the misrecognized obstacle isdeleted during the driving path, and may control the deleted area toblink for a predetermined time period. In the instant case, the deletedarea may be displayed through box processing using a rectangle or thelike, rounding processing of a circle, or the like. Furthermore, theprocessor 114 may mark or block an area in which vehicle driving isimpossible in the driving path.

When the driving path is displayed based on augmented reality on theinterface device 113, the processor 114 may box-process and display afront target for maintaining an inter-vehicle distance with a vehicle infront, and may distinguish and display a line thickness or a line colorof the box of the front target during the normal driving mode and thestop control mode. Furthermore, the processor 114 may display a frontsignal condition (e.g., a traffic light) together during the stopcontrol mode.

The processor 114 may display the area of the misrecognized obstacle inthe driving path, and may display a line color or thickness of the areaof the misrecognized obstacle by distinguishing it from that in thenormal driving mode. In the instant case, the area of the misrecognizedobstacle may be displayed through a frame, box processing such as arectangle, and a rounding process such as a circle of the misrecognizedobstacle.

When the misrecognized obstacle is double-touched by the driver, theprocessor 114 may recognize it as a deletion request, and may transmitthe information related to the misrecognized obstacle to the controlsystem.

When receiving no request for deleting the misrecognized obstacle thatcauses the situation in which the driving of the driving path isimpossible from the driver, the processor 114 may request remote controlto the control system 200.

When receiving a remote control path for the remote control request fromthe control system, the processor 114 may display the remote controlpath based on the augmented reality.

The processor 114 may display a screen for obtaining approval from adriver or an occupant for the remote control path on the interfacedevice 113.

The processor 114 may follow and control the remote control path whenthe approval for the remote control path from the driver or the occupantis completed.

The processor 114 may transmit unapproved information and an unapprovedremote control path to the control system 200 when the approval for theremote control path from the driver or the occupant is not completed.

The sensing device 120 may include one or more sensors that detect anobstacle, e.g., a preceding vehicle, positioned around the host vehicleand measure a distance with the obstacle and/or a relative speedthereof.

The sensing device 120 may include a plurality of sensors to detect anexternal object of the vehicle, to obtain information related to aposition of the external object, a speed of the external object, amoving direction of the external object, and/or a type of the externalobject (e.g., vehicles, pedestrians, bicycles or motorcycles, etc.). Tothe present end, the sensing device 120 may include an ultrasonicsensor, a radar, a camera, a laser scanner, and/or a corner radar, aLight Detection and Ranging (LiDAR), an acceleration sensor, a yaw ratesensor, a torque measurement sensor and/or a wheel speed sensor, asteering angle sensor, etc.

FIG. 2A illustrates a view for describing a sensing device of anautonomous vehicle according to various exemplary embodiments of thepresent invention, and FIG. 2B illustrates a sensing range of a sensingdevice of an autonomous vehicle according to various exemplaryembodiments of the present invention.

Referring to FIG. 2A, the sensing device 120 may include a front radarmounted on the front of the vehicle, a Light Detection and Ranging(LiDAR), a side LiDAR, a side camera, a corner radar, a high-resolutionLiDAR, a rear camera, a rear LiDAR, etc. Furthermore, referring to FIG.2B, a surrounding situation may be detected through radars, cameras, andLiDARs of the front, rear, and side of the vehicle.

The steering control device 130 may be configured to control a steeringangle of a vehicle, and may include a steering wheel, an actuatorinterlocked with the steering wheel, and a controller configured forcontrolling the actuator.

The braking control device 140 may be configured to control braking ofthe vehicle, and may include a controller that is configured to controla brake thereof.

The engine control unit (ECU) 150 may be configured to control enginedriving of a vehicle, and may include a controller that is configured tocontrol a speed of the vehicle.

When receiving information related to a misrecognized obstacle from theautonomous vehicle 100, the control system 200 displays themisrecognized obstacle on a driving path of the autonomous vehicle 100,may delete the misrecognized obstacles on the driving path of theautonomous vehicle 100 by receiving approval from an operator, and maytransmit the driving path of the autonomous vehicle 100 in which themisrecognized obstacle is deleted to the autonomous vehicle 100.

The control system 200 may include a communication device 211, a storage212, an interface device 213, and a processor 214.

The communication device 211 is a hardware device implemented withvarious electronic circuits to transmit and receive signals through awireless or wired connection, and may transmit and receive informationbased on in-vehicle devices and in-vehicle network communicationtechniques. As an example, the in-vehicle network communicationtechniques may include controller area network (CAN) communication,Local Interconnect Network (LIN) communication, flex-ray communication,Ethernet communication, and the like.

Furthermore, the communication device 211 may perform communication byuse of a server, infrastructure, or third vehicles outside the vehicle,and the like through a wireless Internet technique or short rangecommunication technique. Herein, the wireless Internet technique mayinclude wireless LAN (WLAN), wireless broadband (Wibro), Wi-Fi,Worldwide Interoperability for Microwave Access (WiMAX), etc.Furthermore, short-range communication technique may include Bluetooth,ZigBee, ultra wideband (UWB), radio frequency identification (RFID),infrared data association (IrDA), and the like. For example, thecommunication device 211 may perform wireless communication with thevehicle 100, may receive a remote control request from the vehicle 100,and may transmit an approval request for the remote control path and aremote control command.

The storage 212 may store information received from the vehicle 100, anddata and/or algorithm required for the processor 214 to operate, and thelike.

As an example, the storage 212 may store a vehicle path received fromthe vehicle 100, image data photographed through a camera, a remotecontrol path, a remote control command selected by an operator, and thelike.

The storage 212 may include a storage medium of at least one type amongmemories of types such as a flash memory, a hard disk, a micro, a card(e.g., a secure digital (SD) card or an extreme digital (XD) card), arandom access memory (RAM), a static RAM (SRAM), a read-only memory(ROM), a programmable ROM (PROM), an electrically erasable PROM(EEPROM), a magnetic memory (MRAM), a magnetic disk, and an opticaldisk.

The interface device 213 may include an input means configured forreceiving a control command from an operator and an output means foroutputting an operation state of the control system 200 and resultsthereof. Herein, the input means may include a key button, and mayfurther include a mouse, a keyboard, a touch screen, a microphone, ajoystick, a jog shuttle, a stylus pen, and the like. Furthermore, theinput means may further include a soft key implemented on the display.For example, the interface device 213 may display map information inwhich a driving path of the vehicle, a current position of the vehicle,information related to surrounding objects, etc. are marked based onvehicle data received from the vehicle 100. For example, the interfacedevice may include all communication terminals such as a personalcomputer (PC), a notebook computer, a smartphone, a tablet PC, a pad, apersonal digital assistant (PDA), and a wearable device.

The output means may include a display, and may further include a voiceoutput means such as a speaker. In the instant case, when a touch sensorformed of a touch film, a touch sheet, or a touch pad is provided on thedisplay, the display may operate as a touch screen, and may beimplemented in a form in which an input device and an output device areintegrated.

In the instant case, the display may include at least one of a liquidcrystal display (LCD), a thin film transistor liquid crystal display(TFT LCD), an organic light emitting diode display (OLED display), aflexible display, a field emission display (FED), or a 3D display.

The processor 214 may be electrically connected to the communicationdevice 211, the storage 212, the interface device 213, and the like, mayelectrically control each component, and may be an electrical circuitthat executes software commands, performing various data processing anddeterminations described below.

The processor 214 may process a signal transferred between components ofthe control system 200, and may perform overall control such that eachof the components can perform its function normally. The processor 214may be implemented in a form of hardware, software, or a combination ofhardware and software, or may be implemented as microprocessor.

When receiving information related to a misrecognized obstacle from theautonomous vehicle, the processor 214 displays the misrecognizedobstacle on a driving path of the autonomous vehicle, deletes themisrecognized obstacle on the driving path of the autonomous vehicle byreceiving approval from an operator, and transmits the driving path ofthe autonomous vehicle in which the misrecognized obstacle is deleted tothe autonomous vehicle.

When receiving the remote control request from the autonomous vehicle100, the processor 214 may generate the remote control path based on theinformation received from the autonomous vehicle 100.

When receiving no approval for the remote control path from theautonomous vehicle 100, the processor 214 may generate the remotecontrol path for avoiding the misrecognized obstacle again.

FIG. 3 illustrates an example of a screen displaying an obstacle basedon augmented reality in a normal driving mode in an autonomous vehicleaccording to various exemplary embodiments of the present invention.

Referring to FIG. 3 , the target 301 may be box-processed and displayedto maintain a distance between vehicles in front in a normal drivingsituation, and a driving path 302 may be displayed in the augmentedreality.

FIG. 4 illustrates an example of a screen displaying an obstacle basedon augmented reality in a stop control mode in an autonomous vehicleaccording to various exemplary embodiments of the present invention.

Referring to FIG. 4 , during the stop control mode, i.e., when stoppingis required to maintain the distance between vehicles in front, a target401, which is a reason for the stopping, is box-processed to bedisplayed, and a line color and thickness of the box of the target 301during the normal driving mode of FIG. 3 are differently displayed sothat the normal driving mode and the stop control mode may bedistinguished. In the instant case, the line color and thickness of thedriving path 402 may be distinguished from those of the normal drivingmode, but may be displayed in a same manner. FIG. 3 and FIG. 4illustrate examples of box-processing the targets 301 and 401, but thepresent invention is not limited thereto, and edge portions of thetargets 301 and 401 may be darkly displayed, or may be displayed throughrounding processing such as a circle.

FIG. 5 illustrates an example of a screen displaying an obstacleincluding traffic light information based on augmented reality in a stopcontrol mode in an autonomous vehicle according to various exemplaryembodiments of the present invention.

Referring to FIG. 5 , during the stop control mode, not only a fronttarget 501 for maintaining a distance between vehicles in front but alsoa front traffic light 502 may be displayed together to indicate that thevehicle is stopped by the traffic light 502. Similarly, the line colorand thickness of the driving path 503 may be distinguished from those ofthe normal driving mode, but may be displayed in a same manner.

FIG. 6 illustrates an example of a screen displaying an obstacleincluding traffic light information which displays a misrecognizedtarget based on augmented reality in a misrecognition mode in anautonomous vehicle according to various exemplary embodiments of thepresent invention.

Referring to FIG. 6 , a front misrecognized target 601 may displayedthrough box processing, and the line color and thickness of the box ofthe misrecognized target 601 may be distinguished from those of thenormal driving mode. Similarly, the line color and thickness of thedriving path 602 may be distinguished from those of the normal drivingmode, but may be displayed in a same manner.

FIG. 7 and FIG. 8 illustrate examples of a screen for deleting amisrecognized obstacle depending on a driver request according tovarious exemplary embodiments of the present invention.

Referring to FIG. 7 and FIG. 8 , a driver may delete misrecognizedobstacles 701 and 801 by directly double-touching them, or may selectthe misrecognized obstacles 701 and 801 to request remote control to thecontrol system 200. When the driver selects the misrecognized obstacles701 and 801 to request the remote control to the control system 200,information related to the misrecognized obstacles may be transmitted tothe control system 200. In the instant case, the information related tothe misrecognized obstacles may include a current driving path,positions of the misrecognized obstacles, types and sizes thereof, andthe like. The display device of the vehicle 100 may be implemented basedon a touch screen to select the misrecognized obstacles 701 and 801.

FIG. 9 illustrates an example of a screen on which a misrecognizedobstacle is displayed depending on a driver request in a control systemaccording to various exemplary embodiments of the present invention.

When receiving information related to a misrecognized obstacle fromvehicle 100, the control system 200 enables an operator to grasp asituation at a glance by constructing and displaying a 3D screen basedon a current driving path, a position of the misrecognized obstacle, atype and a size thereof, etc. In FIG. 9 , a point 901 where arecognition error occurred is illustrated.

FIG. 10 illustrates an example of a screen displaying a change in aremote control path based on augmented reality in an autonomous vehicleaccording to various exemplary embodiments of the present invention.

Referring to FIG. 10 , the vehicle 100 may receive a remote control pathfrom the control system 200 to display it on a display device. In theinstant case, areas (e.g., no roads) 1001, 1002, and 1003 where vehicledriving is impossible in the remote control path may be displayed, andwhen a misrecognized obstacle is deleted, a deleted state may bedisplayed by processing it with a box 1004. In the instant case, theareas (e.g., no roads) 1001, 1002, and 1003 where vehicle driving isimpossible and the box 1004 in which the misrecognized obstacle isdeleted may be displayed based on augmented reality such that a drivermay intuitively check them in the driving path through separate colors,hatching, etc. The vehicle 100 displays the box 1004 in which themisrecognized obstacle is deleted in a blinking manner so that thedriver can recognize at a glance that the misrecognized obstacle hasbeen deleted. Furthermore, the areas 1001, 1002, and 1003 in which thevehicle driving is impossible may be marked or blocked.

Furthermore, the vehicle 100 may display a remote control path 1005 andadditionally display an arrow 1006, and may blink or highlight the arrow106 such that the driver can check at glance a portion which is modifiedfrom a driving path before finding a misrecognized obstacle to a remotecontrol path which is generated to avoid the misrecognized obstacle.

Hereinafter, a remote control method for an autonomous vehicle accordingto various exemplary embodiments of the present invention will bedescribed in detail with reference to FIG. 11 . FIG. 11 illustrates aflowchart showing a remote control method for an autonomous vehicleaccording to various exemplary embodiments of the present invention.

Hereinafter, it is assumed that the autonomous driving control apparatus110 of the vehicle 100 of FIG. 1 and the control system 200 performprocesses of FIG. 11 . Furthermore, in the description of FIG. 11 , itmay be understood that operations referred to as being performed by eachsystem are controlled by a processor of each of the systems.

Referring to FIG. 11 , the vehicle 100 starts autonomous driving (S101),and the control system 200 prepares for remote control (S102).

The vehicle 100 determines whether driving of a current path isimpossible due to an external environment during autonomous driving(S103), and when the driving of the current path is impossible, displaysa reason why the current route is impossible to drive, i.e., a reasonthe vehicle is stopped, on a display device based on the augmentedreality (S104).

Next, the vehicle 100 checks whether there is a request for ignoring amisrecognized obstacle from a driver (S105), and when receiving arequest for ignoring the misrecognized obstacle from the driver, thevehicle 100 transmits misrecognition related information and surroundingimages to the control system 200 (S106). In the instant case, themisrecognition related information may include information such as acurrent driving path, a position and size of an obstacle which ismisrecognized on the current driving path, position information of thehost vehicle, etc.

Accordingly, the control system 200 checks whether a request of thedriver to ignore the misrecognized obstacle is received from the vehicle100 (S107), and when the request to ignore the misrecognized obstacle isnot received, waits to receive a remote control request from the vehicle100.

On the other hand, when receiving no request for ignoring themisrecognized obstacle from the vehicle 100, the control system 200displays the information related to the misrecognized obstacle receivedfrom the vehicle 100 such that an operator can see it (see FIG. 9 ), andwhen receiving a request for deleting the misrecognized obstacle fromthe operator after the operator checks the misrecognition relatedinformation (S108), deletes the misrecognized obstacle on the currentdriving path and transmits the path to the vehicle 100 (S109).

On the other hand, when receiving no request for deleting themisrecognized obstacle (condition) from the operator, the control system200 determines that the current driving path is mortified or remotecontrol is required, and waits to receive a remote control request fromthe vehicle 100.

On the other hand, when receiving a path in which the misrecognizedobstacle is deleted from the control system 200, the vehicle 100 followsand controls the path in which the misrecognized obstacle is deleted(S110).

In step S105, when receiving no request for ignoring the misrecognizedobstacle, that is, when the driver does not ignore the misrecognizedobstacle and wants to recognize it as a real obstacle, the vehicle 100determines whether there is a remote control request for avoiding theobstacle from the driver (S111).

When a remote control request is inputted from the driver, the vehicle100 requests remote control to the control system 200 (S112). In theinstant case, the vehicle 100 transmits information such as vehicleposition information (coordinates), vehicle surrounding information(surrounding object information the current changing the color of theetc.), and map information (current path of the vehicle) togethertherewith when requesting the remote control.

The control system 200 generates a remote control path based on theinformation received from the vehicle 100 when receiving the remotecontrol request (S113), and transmits the generated remote control pathto the vehicle 100 (S114).

Accordingly, the vehicle 100 displays the remote control path receivedfrom the control system 200 on a display device based on augmentedreality (S115), and receives approval of the displayed remote controlpath from the driver or an occupant.

The vehicle 100 checks whether the displayed remote control path isapproved by the driver or the occupant (S116), and when the approval iscompleted, the vehicle 100 follows and controls a modified remotecontrol path received from the control system 200 (S120).

On the other hand, when the displayed remote control path is notapproved by the driver or occupant, the vehicle 100 transmits unapprovedinformation and the mortified path to the control system 200 (S117).

Accordingly, the control system 200 checks whether the mortified path isreceived from the vehicle 100 (S118), and when the mortified path isreceived, returns to the above-described step S113 to re-generate theremote control path.

On the other hand, the control system 200 determines that the vehicle100 is being remotely controlled to the corresponding path when themortified path is not received, and terminates the remote control whenit is completed. In the instant case, when the remote control iscompleted, the vehicle 100 transmits the remote control path to thecontrol system 200.

As described above, according to various exemplary embodiments of thepresent invention, when an obstacle recognized by the vehicle 100through the sensing device 120 is misrecognized, the misrecognizedobstacle may be deleted depending on a request of the driver, or remotecontrol may be performed to delete the misrecognized obstacle.Accordingly, the driver of the vehicle 100 may easily correct errorsthat occur during autonomous driving, securing reliability of the remotecontrol, improving commercialization of autonomous driving.

Furthermore, according to various exemplary embodiments of the presentinvention, when the remote control path is changed due to amisrecognized obstacle, the changed remote control path may be displayedbased on augmented reality to enable the driver to intuitively recognizeit.

FIG. 12 illustrates a computing system according to various exemplaryembodiments of the present invention.

Referring to FIG. 12 , the computing system 1000 includes at least oneprocessor 1100 connected through a bus 1200, a memory 1300, a userinterface input device 1400, a user interface output device 1500, and astorage 1600, and a network interface 1700.

The processor 1100 may be a central processing unit (CPU) or asemiconductor device that performs processing on commands stored in thememory 1300 and/or the storage 1600. The memory 1300 and the storage1600 may include various types of volatile or nonvolatile storage media.For example, the memory 1300 may include a read only memory (ROM) 1310and a random access memory (RAM) 1320.

Accordingly, steps of a method or algorithm described in connection withthe exemplary embodiments included herein may be directly implemented byhardware, a software module, or a combination of the two, executed bythe processor 1100. The software module may reside in a storage medium(i.e., the memory 1300 and/or the storage 1600) such as a RAM memory, aflash memory, a ROM memory, an EPROM memory, a EEPROM memory, aregister, a hard disk, a removable disk, and a CD-ROM.

An exemplary storage medium is coupled to the processor 1100, which canread information from and write information to the storage medium.Alternatively, the storage medium may be integrated with the processor1100. The processor and the storage medium may reside within anapplication specific integrated circuit (ASIC). The ASIC may residewithin a user terminal. Alternatively, the processor and the storagemedium may reside as separate components within the user terminal.

The above description is merely illustrative of the technical idea ofthe present invention, and those skilled in the art to which variousexemplary embodiments of the present invention pertains may make variousmodifications and variations without departing from the essentialcharacteristics of the present invention.

For convenience in explanation and accurate definition in the appendedclaims, the terms “upper”, “lower”, “inner”, “outer”, “up”, “down”,“upwards”, “downwards”, “front”, “rear”, “back”, “inside”, “outside”,“inwardly”, “outwardly”, “interior”, “exterior”, “internal”, “external”,“forwards”, and “backwards” are used to describe features of theexemplary embodiments with reference to the positions of such featuresas displayed in the figures. It will be further understood that the term“connect” or its derivatives refer both to direct and indirectconnection.

The foregoing descriptions of specific exemplary embodiments of thepresent invention have been presented for purposes of illustration anddescription. They are not intended to be exhaustive or to limit thepresent invention to the precise forms disclosed, and obviously manymodifications and variations are possible in light of the aboveteachings. The exemplary embodiments were chosen and described toexplain certain principles of the present invention and their practicalapplication, to enable others skilled in the art to make and utilizevarious exemplary embodiments of the present invention, as well asvarious alternatives and modifications thereof. It is intended that thescope of the present invention be defined by the Claims appended heretoand their equivalents.

What is claimed is:
 1. An autonomous vehicle comprising: a displaydevice configured to display a driving path of the autonomous vehicle;and an autonomous driving control apparatus including a processor thatis configured to: display a situation in which driving of the drivingpath is impossible on the display device in augmented reality when thesituation in which the driving of the driving path is impossible occursdue to an external environment during autonomous driving of theautonomous vehicle, transmit information related to a misrecognizedobstacle to a control system when receiving a request for deleting themisrecognized obstacle, and receive a driving path in which themisrecognized obstacle is deleted from the control system and controland follow the received driving path.
 2. The autonomous vehicle of claim1, further including: a sensing device configured to detect thesituation in which the driving of the driving path is impossible.
 3. Theautonomous vehicle of claim 1, wherein the processor is configured todisplay the driving path received from the control system, in which themisrecognized obstacle is deleted, on the display device in theaugmented reality.
 4. The autonomous vehicle of claim 3, wherein theprocessor is configured to display an area in which the misrecognizedobstacle is deleted in the driving path and to control the area to blinkfor a predetermined time period.
 5. The autonomous vehicle of claim 3,wherein the processor is configured to mark or block an area in which avehicle is unable to be driven in the driving path.
 6. The autonomousvehicle of claim 1, wherein when the driving path is displayed on thedisplay device in the augmented reality, the processor is configured to:display an area of a front target for maintaining an in-vehicle distancewith a vehicle in front of the autonomous vehicle, and distinguish anddisplay a line thickness or a line color indicating the area of thefront target during a normal driving mode and a stop control mode of theautonomous vehicle.
 7. The autonomous vehicle of claim 6, wherein theprocessor is configured to display a front signal condition during thestop control mode.
 8. The autonomous vehicle of claim 1, wherein theprocessor is configured to: display an area of the misrecognizedobstacle in the driving path, and display a line color or line thicknessindicating the area of the misrecognized obstacle in a normal drivingmode separately from that a stop control mode of the autonomous vehicle.9. The autonomous vehicle of claim 1, wherein the processor isconfigured to transmit information related to the misrecognized obstacleto the control system in response to a request for deleting themisrecognized obstacle from a driver.
 10. The autonomous vehicle ofclaim 1, wherein when receiving no request for deleting themisrecognized obstacle from a driver, the processor is configured torequest remote control of the autonomous vehicle to the control system.11. The autonomous vehicle of claim 10, wherein when receiving a remotecontrol path for the requesting of the remote control from the controlsystem, the processor is configured to display the remote control pathin the augmented reality.
 12. The autonomous vehicle of claim 11,wherein the processor is configured to display a screen for obtainingapproval for the remote control path by a driver or an occupant on thedisplay device.
 13. The autonomous vehicle of claim 11, wherein theprocessor is configured to follow and control the remote control pathwhen the approval for the remote control path by the driver or theoccupant is completed.
 14. The autonomous vehicle of claim 13, whereinthe processor is configured to transmit unapproved information and anunapproved remote control path to the control system when the approvalfor the remote control path by the driver or the occupant is notcompleted.
 15. The autonomous vehicle of claim 1, wherein the autonomousdriving control apparatus includes: a communication device configured tocommunicate with the control system; and a storage configured to storethe remote control path received from the control system.
 16. A controlsystem comprising: a processor configured to display a misrecognizedobstacle on a driving path of an autonomous vehicle when receivinginformation related to the misrecognized obstacle from the autonomousvehicle, to delete the misrecognized obstacle on the driving path of theautonomous vehicle by receiving approval from an operator, and totransmit a driving path of the autonomous vehicle in which themisrecognized obstacle is deleted to the autonomous vehicle.
 17. Thecontrol system of claim 16, wherein the processor is configured togenerate a remote control path to provide the generated remote controlpath to the autonomous vehicle when receiving a request for remotecontrol of the autonomous vehicle for avoiding the misrecognizedobstacle from the autonomous vehicle.
 18. The control system of claim17, wherein the processor is configured to re-generate a remote controlpath for avoiding the misrecognized obstacle when the approval for theremote control path is not received from the autonomous vehicle.
 19. Aremote control method for an autonomous vehicle, the remote controlmethod comprising: displaying, by an autonomous driving controlapparatus, a situation in which driving of a driving path is impossibleon a display device in augmented reality when the situation in which thedriving of the driving path is impossible occurs due to an externalenvironment during autonomous driving of the autonomous vehicle;receiving, by the autonomous driving control apparatus, a request fordeleting a misrecognized obstacle from a driver; transmitting, by theautonomous driving control apparatus, information related to themisrecognized obstacle to a control system; and following andcontrolling, by the autonomous driving control apparatus, a driving pathin which the misrecognized obstacle is deleted by receiving the drivingpath from the control system.
 20. The remote control method of claim 19,further including: displaying, by the autonomous driving controlapparatus, the driving path in which the misrecognized obstacle isdeleted, wherein the displaying includes displaying an area in which themisrecognized obstacle is deleted, and controlling the area to blink fora predetermined time period.